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Large prospective cohort studies are critical for identifying etiologic factors for disease, but they require substantial 
long-term research investment. Such studies can be conducted as multisite consortia of academic medical centers, 
combinations of smaller ongoing studies, or a single large site such as a dominant regional health-care provider. Still 
another strategy relies upon centralized conduct of most or all aspects, recruiting through multiple temporary 
assessment centers. This is the approach used by a large-scale national resource in the United Kingdom known 
as the "UK Biobank," which completed recruitment/examination of 503,000 participants between 2007 and 2010 
within budget and ahead of schedule. A key lesson from UK Biobank and similar studies is that large studies are 
not simply small studies made large but, rather, require fundamentally different approaches in which "process" 
expertise is as important as scientific rigor. Embedding recruitment in a structure that facilitates outcome determination, 
utilizing comprehensive and flexible information technology, automating biospecimen processing, ensuring broad 
consent, and establishing essentially autonomous leadership with appropriate oversight are all critical to success. 
Whether and how these approaches may be transportable to the United States remain to be explored, but their 
success in studies such as UK Biobank makes a compelling case for such explorations to begin. 

cohort studies; epidemiology; prospective studies 



Large prospective cohort studies are indispensable for 
identifying etiologic factors for disease, but their large scope 
requires substantial long-term research investments. Prospec- 
tive cohort studies are often conducted as multisite consortia 
of large academic medical centers, each responsible for re- 
cruitment, examination, and follow-up of participants in its 
geographic area. This was the model initially explored for 
a prospective study of up to 1,000,000 Americans (1), but its 
large anticipated costs present a substantial hurdle. 

Other models include combining ongoing, smaller studies 
to provide more immediate answers at potentially lower costs 
(2, 3), but existing studies are often limited and variable in the 
diversity of populations, exposures, and diseases ascertained; 
standardization of methods; and adequacy of existing consent 
and data access (4, 5). Large cohorts recruited at a single site, 
often through a dominant regional health-care provider, rep- 
resent another approach (6, 7). Still another strategy relies 



upon centralized conduct of most or all aspects of the study, 
although responsibility for individual study-wide aspects can 
be distributed to collaborating centers (Figure 1) (8). This is 
the approach used by a large-scale national resource in the 
United Kingdom known as "UK Biobank" (9), initiated in 
2004 to examine genetic and environmental risk factors for 
complex diseases. Briefly, UK Biobank issued mailed in- 
vitations to about 9 million persons aged 40-69 years living 
within 25 miles (40.23 km) from the 21 study assessment 
centers and registered with the National Health Service in the 
United Kingdom. More than 500,000 participants responded 
to this invitation and were examined between April 2007 and 
June 2010. Detailed questionnaire, interview, and measure- 
ment data were obtained during the 2-3 hour baseline as- 
sessment, and multiple samples of blood, saliva, and urine 
were collected (Table 1). Repeat assessments on a subsample 
of 20,000 participants will address regression-dilution bias, 
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Figure 1. Traditional distributed model contrasted with novel centralized model. In A (a traditional distributed model), the roles of collaborating, 
typically academic, centers in distributed models include large numbers of tasks that could be located in one or more central units. Coordinating 
centers in these models tend to be responsible for developing and implementing data collection and monitoring systems, training and certification 
standards for study staff, and the biospecimen repositories. In B (a novel centralized model), centralized models concentrate multiple study-wide 
activities in one or more coordinating centers, with each potentially handling related clusters of activities. Assessment centers focus on participant 
examination and transmission of data and specimens, and they are kept open only so long as they are productive. Successive waves of assessment 
centers represented by an initial group of centers 1-6 are followed by a wave of centers 7-12, and so on. 



and health outcomes will be assessed primarily through 
national health records. 

The centralized model, which UK Biobank adopted after 
rejecting a costly, decentralized approach, has enabled its 
leaders to achieve exceptional efficiencies in recruitment, 
assessment, and record linkage while retaining diversity in 



demographics and exposures. Cohort recruitment was achieved 
not only ahead of schedule but also within budget (roughly 
$100 million for 503,000 participants), enabling substantial 
enhancements to be added to the study protocol. 

The evident success of the UK Biobank model may pro- 
vide valuable lessons for the conduct of large-scale studies 
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Table 1. Components of UK Biobank, With Participant Recruitment between 2007 and 2010 



Baseline Questionnaire 



Baseline Physical Measurements 



Follow-up and Future IVIeasures 



Sociodemographic 
Family history 

Psychosocial 

Environmental 

Lifestyle 

Cognitive function 
Health status 
Food frequency 

Internet-administered 24-hour dietary 
questionnaire 



Blood pressure 
Weight, body impedance 

Waist and hip circumferences 
Seated and standing heights 
Grip strength 
Spirometry 
Bone density 

Mailed triaxial accelerometers 

Enhanced phenotyping (last 

100,000-150,000 participants recruited) 

• Hearing 

• Vascular reactivity 

• Visual acuity 

• Refractive error 

• Intraocular pressure 

• Corneal biomechanics 

• Optical coherence tomography 

• Fitness assessment 



Stored blood, urine, saliva 

Repeat baseline assessment 
(20,000 participants) 

Access national health records 

• Death 

• Cancer 

• Hospitalizations 

• Primary care 



Abbreviation: UK, United Kingdom. 



that may or may not be feasible to implement in the United 
States. The National Institutes of Health thus convened 
a symposium summarized in this paper, involving several 
large studies (Table 2) (10), to 1) examine novel aspects of 
the UK Biobank design and its strengths and weaknesses; 
2) compare the UK Biobank approach with those of other 
large studies; and 3) identify lessons learned in maximizing 
the efficiencies of these studies. 

OVERARCHING CONSIDERATIONS 

Suggested characteristics of an optimal cohort study for 
examining genetic and environmental influences on disease 
have been described (Table 3) (1). Large size is a key com- 
ponent, as relevant genetic variants and other risk exposures 
may be uncommon and effect sizes are often modest (11). 
These are not, however, simply small studies made large, as 
the costs and inefficiencies in 100-fold expansion of a 5,000- 
person, disease-specific cohort study are prohibitive. Large 
studies require fundamentally different approaches in which 
minimizing cost is a primary consideration and "process" 
expertise to maximize efficiency of high-throughput operations 
is as important as scientific rigor. Modern industrial design 
principles that identify and manage critical choke points 
are essential to ensuring high throughput and maintaining 
quality (12). 

Decentralized models involving semipermanent research 
centers can be expensive to maintain and can present chal- 
lenges in standardization. Using temporary assessment centers 
in a centralized model avoids the need to maintain remote 
offices, staff, and laboratory capabilities. Centralized models 
may provide greater overall control of costs, as well as agility 



in responding to changing situations such as relocating under- 
performing sites or modifying suboptimal procedures. They 
may thus free investigators to focus on science rather than 
miring them in the operational concerns of their individual 
sites. 

Centralized models may also have drawbacks. The inherent 
need to choose a specific population base and standardized 
assessments for a very large, centralized cohort may limit 
the questions that can be addressed. This contrasts with the 
diverse approaches fostered by multiple independent studies 
that can be a powerful force for improving methodology and 
assessing the replicability of findings. In addition, the potential 
for disenfranchising academic centers accustomed to oper- 
ational leadership in their assigned geographic area may risk 
losing critical scientific input from these groups. Special care 
is also needed to involve community-based organizations and 
to ensure that they feel local concerns are being addressed in 
a centralized study design. Keeping them engaged in a cen- 
tralized model requires significant effort with frequent local 
visits and community meetings that include the study leader- 
ship. Being chosen as part of a major national effort can be 
a source of considerable community pride, especially if it is 
clear that community input is valued and implemented. 

Experienced investigators may also have well-functioning 
local recruitment systems and understanding of unique local 
conditions that may require tailoring of methods. Approaches 
for harnessing this expertise need careful attention, but they 
might include engaging academic investigators in protocol 
development and implementation or tasking individual aca- 
demic centers with study-wide functions such as ensuring 
diversity of participants, developing novel substudies, or 
responding to queries through a participant call center. 
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Table 2. Large Studies Examined 



Decentralized, Multisite 



Single Site 



Centrally Coordinated 



Canadian Partnership for Tomorrow 

European Prospective Investigation into 
Cancer and Nutrition (EPIC) 

National Children's Study 

Prostate, Lung, Colorectal, and Ovarian 
(PLCO) Cancer Screening Trial 

VA Genomic Medicine Program 

Women's Health Initiative 



Kaiser Permanente Research Program on 
Genes, Environment, and Health (RPGEH) 

Marshfield Clinic Personalized Medicine 
Research Program (PMRP) 

Vanderbilt BioVU, a research resource providing 
a "view into biology" at the level of DNA 



American Cancer Society Cancer 
Prevention Study 3 (ACS CPS-3) 

LifeGene 



National Health and Nutrition 
Examination Survey (NHANES) 

UK Biobank 



Abbreviation: UK, United Kingdom. 



A key aspect of limiting costs is the vigor with which a high 
"response rate" is pursued during recruitment. Nationally 
representative surveys, such as the National Health and 
Nutrition Examination Survey (NHANES) (13) and smaller 
disease-specific studies such as the Cardiovascular Health 
Study (14), serve important aims in providing population- 
based estimates of disease prevalence and incidence. For 
this purpose, they need representative population samples, 
necessitating considerable expenditures to ensure high re- 
sponse rates (15, 16). The limitations of an essentially vol- 
unteer sample are well known, particularly the generally 
healthier profile, higher educational attainment, and greater 
health consciousness of volunteers (17). This "healthy volun- 
teer" effect can lead to underestimation of disease prevalence 
and incidence, but its impact on relative risk estimates for 
environmental and genetic factors is generally not impor- 
tant (18, 19). Although high response rates are critical for 
population-based estimates of disease incidence, preva- 
lence, or mortality, a 10% or even 1% response rate may be 
acceptable in certain situations, especially if the focus is on 
risk associations and the base population is large enough to 
capture a diversity of exposures and backgrounds. Results 
from such studies can still be applicable to populations with 
different distributions of these exposures, although this 

Table 3. Characteristics of Optimal Cohort Study'' 

• Large in scale (hundreds of thousands of participants) 

• Diverse regarding age, race/ethnicity, socioeconomic status, 
geographic region 

• Address multiple diseases/risk factors 

• Highly efficient recruitment, data collection, sample processing 

• Standardized or harmonized terminology to facilitate 
interoperability with other data 

• Linked personal electronic records and biospecimens 

• Broad content and high quality of samples and data 

• State-of-the-art technology for environmental sampling, 
laboratory methods, genomics, information technology 

• Cost effective 

• Data available for qualified researchers 

'' According to F. S. Collins {Nature. 2004;429(6990):475-477) (1). 



cannot be proven but only assumed for exposures that 
are unknown or unknowable. High response rates thus need 
not be a driving factor scientifically or economically. For 
these reasons, UK Biobank chose to emphasize diversity but 
to de-emphasize response rates. It has accepted yields of 
5%-10% while realizing substantial savings by not attempt- 
ing to convert initial refusals, as conversion was not found to 
be effective in pilot studies. 

Other important cost determinants include method of as- 
certainment (such as registry vs. household enumeration), 
complexity of data collection, and follow-up methods (such 
as active vs. passive). Choice among these is largely driven 
by the scope and goals of a given study. Irrespective of these 
choices, however, centralized approaches are hkely to provide 
the advantages described above, as evidenced by the marked 
reductions in UK Biobank costs when it shifted to a centralized 
design while keeping other aspects constant. 

RECRUITMENT 

Embedding participant recruitment within a structure that 
facilitates determination of study outcomes, such as an ex- 
isting electronic medical records system (20) or a reimburse- 
ment system (16), greatly simplifies participant follow-up. 
The Kaiser Research Program on Genes, Environment, and 
Health (RPGEH), Marshfield Chnical Personalized Medicine 
Research Program (PMRP), UK Biobank, and Vanderbilt 
University BioVU all recruit from their respective patient 
registries, simplifying subsequent record linkage for follow-up. 
As it can be challenging to use such information to address 
research hypotheses, such data may need further review and 
validation. 

Centralizing the invitation and appointment system can 
maximize efficiencies in recruitment and scheduling, allow- 
ing study-wide monitoring of invitations sent, confirmations 
received, and appointments made. A centralized approach 
also facilitates weekly or even daily monitoring of key char- 
acteristics of participants study-wide; sampling fractions can 
then be modified in real time to increase recruitment of under- 
subscribed groups. 

Recognizing that participation rates tend to fall with in- 
creased distance to an examination center, invitations can 
be concentrated close to the center and extended gradually 
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outward until participation falls enough to justify closing that 
center and opening another in a new area (Web Figure 1, 
posted on the Journal\ website (http://aje.oxfordjournals. 
org/)). High-performing centers can also be kept open longer 
than originally planned as long as they remain cost-effective 
and provide sufficient diversity in exposures and demograph- 
ics. Optimal locations for assessment centers can be pinpointed 
by mapping population density, transportation routes, and 
available office space. Establishing assessment centers where 
and when they are needed, staffing them primarily with tem- 
porary personnel specifically trained for the needs of the pro- 
ject, and closing them when they cease to meet recruitment 
goals provide considerable efficiencies over establishing a fixed 
number of centers and keeping all of them open throughout the 
recruitment period. Cost per participant recruited is a driving 
force in the cost of these studies and should be a primary focus 
in designing recruitment strategies, as long as fundamental 
scientific goals remain paramount. 

Novel approaches to recruitment include enrolling partic- 
ipants at fund-raising events, such as the American Cancer 
Society's "Relay For Life" or through worksite recruitment 
at corporate partners of such organizations. Yield of these 
strategies can be low, however, particularly for certain demo- 
graphic groups. Direct mail approaches may also introduce 
some demographic biases because willingness to respond to, 
or indeed even to open, mailed invitations varies consider- 
ably. Even a 1 % yield, however, as experienced in some US 
studies (21), may be cost-effective if mailing is inexpensive 
and there is a sufficiently large and diverse base population. 
Internet and other social media approaches have potential 
for even lower costs and are being explored in many studies, 
but they do have pronounced differences by demographics; 
this may abate over time as Internet use expands. Text mes- 
saging for confirmation of appointments, for example, has 
proven to be effective with younger participants (22). 

Another aspect that can be usefully centralized is respond- 
ing to queries about the study. A single participant call center 
can rapidly gain experience in responding to concerns effec- 
tively and consistently, and it can also allow rapid escalation 
to more senior staff as needed. In UK Biobank, for example, 
roughly 50% of invitees calling to express concerns ultimately 
participated, and data on specific concerns were used to modify 
invitation materials and clinic flow (23). 

Sending invitees a provisional appointment (". . . an ap- 
pointment has been scheduled for you on date/time; if you 
would like to confirm or change this optional appointment 
please contact us . . .") entails some risk, as they may view 
it as presumptuous. It has, however, been used successfully 
by some health services for their national screening programs, 
such as National Health Service breast screening in the United 
Kingdom. Like all other aspects of recruitment, it needs to be 
carefully piloted, and its effectiveness may well vary by cul- 
tural or demographic characteristics. Still, it was quite success- 
ful in UK Biobank where roughly half of all participants who 
attended did so at the time of their provisional appointments. 

QUESTIONNAIRE AND EXAMINATION 

Several studies, including the European Prospective Inves- 
tigation into Cancer and Nutrition (EPIC), the US National 



Health and Nutrition Examination Survey, and UK Biobank, 
have recognized the efficiency of having participants re- 
spond to questionnaires by touch screen rather than through 
interviewer- or self-administered forms. Well-designed touch- 
screen systems can allow a single staff member to assist 15-20 
participants at a time. Electronic data capture in real time 
minimizes errors in transcription, and ongoing analysis of 
data quality can help to identify and correct problems early. 
In addition, computerized instruments can be used to collect 
more than questionnaire data, including tests of cognitive func- 
tion, audiology, and simple motor function. Internet-based 
24-hour dietary recalls administered remotely in UK Biobank, 
for example, will permit more in-depth evaluation of intake 
of total energy and some nutrients and will facilitate repeat 
measurements. 

BIOREPOSITORY 

Biorepositories provide much of the future value of cohort 
studies, as they permit measurement of biomarkers of expo- 
sures or intermediate phenotypes often not even imagined at 
the time the study is initiated. Central processing of biospeci- 
mens typically offers increased consistency and achievable 
throughput, a robust data trail, and lower costs compared with 
local processing, although this must be piloted carefully in 
each setting (12). Automated, industrial-scale specimen pro- 
cessing and storage systems are indispensable in large studies 
although their costs may make them impractical for smaller 
studies. Implementation following best practices used in the 
manufacturing industry can reduce project risk and build in 
quality and robustness (12). 

HEALTH OUTCOMES 

Large-scale studies can examine a nearly unlimited number 
of outcomes provided the outcomes can be classified reliably 



Table 4. Key Lessons From New Models of Large Cohort Studies 

■ Ensure that future studies, including disease-specific studies, 
address the widest possible range of outcomes to permit 
combining data for increased study power 

■ Use standardized or harmonized (not identical but comparable) 
measures to permit diverse studies to be combined^ 

■ Establish consents that allow for broad data sharing as the norm 

■ Maximize cost-efficiency where appropriate by 

• Exploring centralized recruitment and examination models 

• Considering lower recruitment yield if associations rather 
than prevalence are the primary objective 

• Utilizing electronic records 

• Emphasizing industrial-scale process expertise as the driver 
of process organization, implementation, and monitoring 

• Maximizing the capabilities of information technology to 
ensure high-quality data, rapid transfer, and real-time 
monitoring 

• Phasing activities to be completed only shortly before they 
are needed 

^ According to DataSHaPER (http://www.datashaper.org/) (30) and 
C. M. Hamilton et al. (doi:10.1093/aje/kwr193) (31). 
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and sufficient numbers of cases are available. Collection of 
outcome data is greatly facilitated by comprehensive elec- 
tronic medical records; indeed, the feasibility of UK Biobank 
was questioned before links with National Health Service 
outcomes data were firmly established. Electronic medical 
records can pose unique challenges, however, because they 
often include complex and variable medical terminology, 
may be difficult to interpret with regard to timing of measure- 
ments, and cannot easily be reduced to common elements for 
pooled analyses. 

To the degree that standardized terminology can be ex- 
tracted from electronic medical records, however, automated 
algorithms can be developed and are being actively used 
in programs such as the Marshfield and Vanderbilt studies 
(24, 25). Extending electronic systems of outcome assessment 
to a national scale may be one of the greatest challenges to 
implementing a truly nationwide US cohort study. 

INFORMATION TECHNOLOGY 

High-quality information technology systems arguably 
represent the single most important infrastructure compo- 
nent for ensuring high data quality and cost control in large 
studies. Ideally, they should be designed to maximize par- 
ticipant throughput, perform routine data quality checks, and 
enable access and linkage to other data systems. 

Real-time data capture and essentially daily review are 
critical to the smooth operations of large-scale projects, as 
the volume of data and potential errors identified through 
less frequent monitoring could become overwhelming. Use 
of commercial software when possible may be cost-effective, 
particularly for routine components such as laboratory infor- 
mation systems, but such systems must be interoperable with 
other assessment center systems. Working within established 
electronic medical record systems can facilitate leveraging 
extensive infrastructures established for clinical care. Research 
use of these systems may also facilitate further development 
of electronic medical records for research and, potentially, for 
direct incorporation of research findings into clinical care. 

Threats to participant privacy may best be managed by 
removing obvious identifiers and establishing user agreements 
cosigned by an outside data user's institution, with reliance 
on professional standards of conduct to prevent misuse 
(26). Minimizing reidentification risk is an area of active 
investigation (27). 

CONSENT AND COMMUNITY CONCERNS 

Maximizing the value of large-scale studies requires ready 
access by a wide community of investigators for research 
consistent with participants' consent. Studies not requiring 
a high recruitment yield can include broad data collection, 
sharing, and use in their initial consent process and exclude 
invitees uncomfortable with these terms. Other studies may 
allow multiple levels of consent designating permitted research 
uses and data acquisition, and use can then be customized to 
the level of consent. Funders of these studies also have a re- 
sponsibility, and a clear interest, in maximizing the value 
of the research investment by ensuring broad data sharing 



within the constraints of participant privacy and consent. 
Participants' expectations for return of research results may 
also need to be addressed (28). 

External oversight helps to ensure that participants' con- 
cerns are voiced and addressed. Many such studies have 
community advisory boards that meet regularly, while larger 
studies may have more formally chartered and even nationally 
recognized advisory groups such as UK Biobank's indepen- 
dent Ethics and Governance Council. Such groups essentially 
"hold up a mirror" to ongoing studies and provide valuable 
input as well as wider credibility on issues such as privacy, 
adherence to consent limitations, and ethics. Funders should 
maintain some distance from ethics and governance issues, 
yet cannot remain totally aloof, as large studies generally 
carry the imprimatur of the funding organization and potential 
lapses will often be laid at their door. 

CONCLUSIONS AND POTENTIAL NEXT STEPS 

That large prospective studies are not simply small studies 
made large is a crucial consideration in successfully carrying 
them out. The scale of these studies necessitates entirely dif- 
ferent approaches, emphasizing cost-efficiency to a degree that 
may appear inimical to the scientific method. In fact, it can 
be indispensable to the science, enabling studies to go forward 
that might otherwise be impossible. The £68 million (~$100 
million) cost of developing and recruiting the UK Biobank 
cohort over 3 years stands in stark contrast to the roughly $400 
million per year for 10 years estimated for traditional models 
of conducting a large cohort study in the United States (29). 

Cohort studies, whether large or small, have much to learn 
from and contribute to each other (Table 4). As all models 
have their strengths and weaknesses, it is unwise to propose 
from among them the one "best," but better to recognize the 
need for, and complementary nature of, a variety of designs 
and approaches. Adoption of a large, centralized model for 
one study should not, by virtue of its size and cost, have 
a chilling effect on enthusiasm for and funding of other 
complementary approaches. 

Methods successfully used in European studies, such as the 
European Prospective Investigation into Cancer and Nutrition, 
LifeGene, and UK Biobank, may not all be directly transport- 
able to the United States, where infrastructures for facilitating 
follow-up on a national scale are limited and where diversity 
in lifestyle, ancestral origin, and geography, as well as sheer 
size, is considerably greater than in many other countries. 
Careful piloting and critical evaluation will be needed to 
assess the feasibility of such models in the United States. 
The success to date of these models in studies such as 
UK Biobank, however, makes a compelling case for such 
explorations to begin. 
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